Search for: All records

An extensively studied phenomenon of the past few years in training deep networks is the implicit bias of gradient descent towards parsimonious solutions. In this work, we further investigate this phenomenon by narrowing our focus to deep matrix factorization, where we reveal surprising low-dimensional structures in the learning dynamics when the target matrix is low-rank. Specifically, we show that the evolution of gradient descent starting from arbitrary orthogonal initialization only affects a minimal portion of singular vector spaces across all weight matrices. In other words, the learning process happens only within a small invariant subspace of each weight matrix, despite the fact that all parameters are updated throughout training. From this, we provide rigorous justification for low-rank training in a specific, yet practical setting. In particular, we demonstrate that we can construct compressed factorizations that are equivalent to full-width, deep factorizations throughout training for solving low-rank matrix completion problems efficiently. 

Early identification of rotating machinery faults is crucial to avoid catastrophic failures upon installation. Contact-based vibration acquisition approaches are traditionally used for the purpose of machine health monitoring and end-of-line quality control. In complex working conditions, it can be difficult to perform an accurate accelerometer based vibration test. Acoustic signals (sound pressure and particle velocity) also contain important information about the operating state of mechanical equipment and can be used to detect different faults. A deep learning approach, namely one-dimensional Convolution Neural Networks (1D-CNN) can directly process raw time signals thereby eliminating the human dependance on fault feature extraction. An experimental research study is conducted to test the proposed 1D-CNN methodology on three different electric motor faults. The results from the study indicate that the fault detection performance from the new acoustic-based method is very effective and thus can be a good replacement to the conventional accelerometer-based methods for detection and diagnosis of mechanical faults in electric motors.